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A goal in the forensic interpretation of scientific evidence is to 
make an inference about the source of a trace of unknown origin. 

The evidence is composed of the following three elements: (a) the 
trace of unknown origin, (b) a sample from a specific source, and 
(c) a collection of samples from the alternative source population. 

The inference process usually considers two propositions. The first 
proposition is usually referred to as the prosecution hypothesis and 
states that a given specific source is the actual source of the trace 
of unknown origin. The second, usually referred to as the defense 
hypothesis, states that the actual source of the trace of unknown ori¬ 
gin is another source from a relevant alternative source population. 

One approach is to calculate a Bayes Factor for deciding between the 
two competing hypotheses. This approach commonly assumes that 
the alternative source population is completely known or uses point 
estimates for its parameters. Contrary to this common approach, we 
propose a development that incorporates the uncertainty on the al¬ 
ternative source population parameters in a reasonable and coherent 
manner into the Bayes Factor. We will illustrate the resulting effects 
on the calculation of several Bayes Factors for different situations 
with a well-studied collection of samples of glass fragments. 


1. The Identification of Source Problem. The goal of a forensic scientist is help decide 
between two competing forensic hypotheses, one presented by the prosecution, denoted H p , and 
one by the defense, denoted Hy. Sampling models corresponding to each of the forensic hypotheses 
are denoted M p and My, respectively. The statement of the forensic hypotheses and the sampling 
models depends upon the source identification question being asked to the scientist. The specific 
source identification question considers whether the trace originates from a fixed specific source. 
For the specific source identification problem, the corresponding forensic hypotheses and sampling 
model statements are given below * 1 . 

Forensic Hypotheses 2 

H p \ The trace originated from the specific source. 

Hd'. The trace did not originate from the specific source, but from another source in the 
relevant alternative source population. 


Primary 62C10; secondary 62F15 
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1 The specific source identification problem can be contrasted with the common source identification problem which 
seeks to answer the question of whether or not two traces of unknown origin share a common source. In the common 
source case, the source is not fixed. 

J The forensic hypotheses for the common source problem are given by 
H p \ The two traces originated from the same source. 

Hd'. The two traces originated from two different sources. 
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Sampling Models 3 

M p : The trace and the control samples were both generated by the specific source. 

M ( f The trace was not generated by the specific source, but generated by some other ran¬ 
domly selected source in the relevant alternative source population. 

Two types of information exist in a case to support either the defense hypothesis or the pros¬ 
ecution hypothesis, quantifiable and conceptual [Aitken and Taroni (2004)]. The conceptual in¬ 
formation includes relevant background and circumstantial information, and is denoted /. All the 
pieces of quantifiable information gathered in relation to a specific trace are collectively called the 
evidence, denoted E. For statistical purposes, E is a set of random elements, whereas I is a set of 
constraints that determine the form of the sampling models for E. 

The evidence in the specific source problem is composed of three elements, E = {E u , E s , E a }, 
where E u denotes a set of observations made on objects from an unknown source, E s denotes a set 
of observations made on objects from a fixed specific source, and E a denotes a set of observations 
made on objects from a relevant population of possible alternative sources. For the specific source 
models, we are assuming that E u , E s . and E a are three independent samples drawn in the following 
way: 

1. E s is a simple random sample from a given specific source determined by M p . Let 9 S denote 
the parameters necessary to describe this sampling induced distribution. 

2. E a is constructed by first taking a simple random sample of sources from a given relevant 
population of alternative sources; then from each sampled source we have a simple random 
sample. Let 9 a denote the parameters necessary to describe this sampling induced distribution. 

3. E u is a simple random sample from a single source. It is unknown whether the source of E u 
is the specific source determined by M p or if E u typically arises from a randomly selected 
source in the relevant population of possible alternative sources. The sampling distribution 
of E u is characterized by either the parameters 9 S under M p or 9 a under Mj. 

Given the background information I, it is assumed that E s contains no further information 
about 9 a and conversely, that E a contains no further information about 9 S . For the purpose of this 
paper, we will limit ourselves to the models described above. When the specific source identification 
problem deviates from the models described above, for example when 9 a only contains information 
on the means of the sources in the alternative population, then the statistical methods will tend to 
be more complicated than those that follow for this development. 

In most approaches used to compare the prosecution and defense models, point estimates are 
used for parts, or all, of the parameters describing the distribution of the evidence. In particular, 
several authors suggest to calculate Bayes Factors for deciding between the two models [Aitken and 
Stoney (1991)]. However, they assume that 9 a is known, or they estimate it from the data within a 
nominal degree of uncertainty. This approach leads to a Bayes Factor that is intractable and often 
unfeasible to calculate, especially when the evidence is high-dimensional. In this paper, we propose 
a factorization of the Bayes Factor that accounts for the uncertainty on the 9 a in a reasonable and 
coherent manner and which can be calculated in practical situations. 

J The sampling models for the common source problem are given by 

M p \ The two traces were generated by the same randomly selected source in the relevant alternative source popu¬ 
lation. 

Md : The two traces were generated by two different randomly selected sources in the relevant alternative source 
population. 
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2. Notational Conventions. We use the following conventions for distinguishing between 
sampling-induced probability and probability used as a measure of belief: 

1. Latin letters denote sampling induced probability measures; for example, f(e s \9 s ) denotes 
the likelihood of observing the realized value of the sample from the specific source given the 
actual value of the specific source distribution parameters. 

2. Greek letters denote a probability measure that is a measure of belief; for example, ir(9 s \e s ) 
denotes the posterior density of 9 s \e s , which describes our belief about the value of 6 S after 
observing a sample E S \6 S . 

3. When combining a belief with a sampling induced probability through Bayes theorem, the 
result is another belief that is informed or updated by the observed sample. We denote the 
resulting distribution with a vr. 

In this setting, the stochastic nature of the evidence E is characterized by an unknown but 
fixed set of parameters 0. However, 6 is usually of interest only in so far as knowledge of its 
value facilitates the quantification of support that E provides for either the prosecution hypothesis 
or the defense hypothesis. In this sense, having to estimate 9 is a nuisance, and hence in the 
statistical nomenclature these parameters in this situation are known as ‘nuisance parameters’. 
The application of Bayesian methods to our problem requires us to specify priors for these nuisance 
parameters; one summarizing our belief about how the specific source generates evidence, ^t( 9 s ), and 
another summarizing our prior belief about how the alternative source population stochastically 
generates evidence, n(9 a ). 

3. Statistical Methods. 

3.1. Introduction to the Bayes Factor. Dating back to the 1970’s, the specific source identifi¬ 
cation problem has been approached within the context of subjective Bayesian forensic hypothesis 
testing (See Aitken and Stoney (1991), Lindley (1977), and Shafer (1982) for more details). Histor¬ 
ically, the specific source identification problem has been limited to applications in which the data 
is inherently low dimensional, the stochastic nature of the evidence can be characterized by a com¬ 
mon parametric family of distributions, and the evidence from the alternative source population is 
sufficiently precise that it completely characterizes the stochastic nature of the alternative source 
population. By approaching the specific source identification problem as a Bayesian hypothesis test, 
the forensic statistician is tasked with providing a summary of the scientific evidence that is logical 
and coherent for updating a prior belief structure concerning the two competing hypotheses. The 
summary is typically known as a Bayes Factor [Good (1991)]. Traditionally, this summary is used 
within the context of a Bayes’ rule as follows: 

P(H p \e,I) _ P (e\H p , I) P (H P \I) 

{ ' 1 P (H d \e, I) P(e\H d ,I ) P (H d \I)’ 

v ^ v ^ v ^ y 

Posterior Odds Bayes Factor and/or Prior Odds 
Likelihood Ratio 

where e is the realization of the evidence, H p is the prosecution hypothesis, H d is the defense 
hypothesis, and I is the relevant background information common to both hypotheses. The prior 
odds summarize our relative belief concerning the validity of the prosecution and defense forensic 
hypotheses before observing the evidence. 

The Bayes Factor then allows us to update our belief following the observation of the evidence 
and arrive at the posterior odds concerning the relative validity of the two hypotheses. If the Bayes 
Factor (and the corresponding posterior odds) is sufficiently high, then we support the prosecution 
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hypothesis; on the other hand if it is sufficiently close to zero, we support the defense hypothesis. In 
effect the Bayes Factor is providing a numerical summary of the answer to both of these questions: 

“What, is my belief about the likelihood of observing the evidence under the prosecution hypothesis?” 

versus 

“What is my belief about the likelihood of observing the evidence under the defense hypothesis?” 

It is extremely important note that the use of a Bayes Factor in the context of formal Bayesian 
model selection, two sets of probability measures are required. The first is the prior beliefs concern¬ 
ing the relative validity of the two competing models, which has been described as the prior odds 
in Equation 3.1. The second is a set of priors that characterizes the belief about the parameters for 
the stochastic generation of the evidence under the prosecution and defense models. Since the main 
focus of this paper is to study the Bayes Factor for the specific source identification problem under 
certain conditions, we will only discuss the second set of priors for characterizing the parameters 
of the sampling models. 

3.2. Known Alternative Source Population Parameters. As an introduction to derivations asso¬ 
ciated with the Bayes Factor for the specific source identification problem, we will derive the value 
of evidence as presented by Lindley (1977). In this section we are assuming that we have a well- 
studied alternative source population with known parameters, 9 aQ - The only unknown parameters 
that are contributing to the uncertainty about the value of the evidence are the ones associated 
with the specific source, 6 S . Let e = {e s , e u , e a } represent the realization of the random element E 
for a specific case at hand. Let 9 = {#<,, 0 aQ } and n(9) = tt( 9 s ) be a probability distribution used 
to describe our prior belief about 9 since there is no uncertainty about 9 aQ ■ 

Define the value of the evidence as 


(3.2) 


V ss (e) = 


n(e\H P ,I) 
7r(e| H d ,I)' 


Computing the value of evidence in this form involves evaluating the likelihood of the entire set of 
evidence e. This can be computationally intensive and often unfeasible. To obtain a computationally 
tractable form of the value of evidence, Aitken and Taroni (2004) have proposed a factorization 
of V ss (e) which assumes that 9 a is known or can be estimated from the data. The factorization 
develops as follows: 


J f(e\9,M p )n(9\M p )d9 
f f(e\9,M d )ir(9\M d )d9 
I f(e u \0, M p )f(e s \9 , M p )f(e a \9, M p )-jr(9)d9 
f f(e u \9, M d )f(e s \9, M d )f(e a \9, M d )n(9)d9 
f f(e u \9 s )f(e s \9 s )f(e a \9 ao M9 s )d9 s 
Jf(e u \9 ao )f(e s \9 s )f(e a \9 ao )7T(9 s )d9 s 
f{ea\9a 0 )ff{e u \9 s )f(e s \9 s )ir(9 s )d9 s 
f(e u \9 aQ )f(e a \9 ao )ff(e s \9 s )7T(9 s )d9 s 
1 J f(e u \9 s )f(e s \9 s )7T(9 s )d9 s 

f(e u \9a 0 ) X J f(e s \9 s )n(9 s )d9 s 


see Note 1 below 
see Note 2 below 


see Note 3 below 


see Note 4 below 


see Note 5 below 
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f(e u \0a o ) 


f(e u \O s ) 


f(e s \6 s )Tr(6 s ) 
f f(e s \9 s )ir(6 s )dd s 


f f(e u \9 s )Tr(0 s \e s )d9 s 
f(e u \9a 0 ) 
Tr(e u \e s , M p , I) 
f(e u \0a o ) 


see Note 6 below 
see Note 7 below 


Note 1. We can drop the conditional notation on / since the background information will be 
the same for both the prosecution and the defense and the relevant information has been considered 
in the models. 


Note 2. The definition of the marginal belief of X given some parameter <p is ir(x) = / /(x|</>)7r(</>)d^>. 

Note 3. f(e\9) is the likelihood function for observing e. Therefore, f(e\6) = f (e u \9) f (e s \9) f (e a \9). 

Also, 7 t(0) does not depend on M p or M ( i so the conditional notation can be dropped. 


Note 4. By definition of 7r(0) = vr(0 s ). The parameters for E s and E a are hxed so they will be 
the same for both M p and Mj. Under the prosecution model, M p , E u is characterized by 9 S since the 
prosecution believes the specific source is the origin of the trace. Therefore, f(e u \9,M p ) = f{e u \9 s ). 
Under the defense model, M f ). E u is completely characterized by 64 () since the defense believes the 
trace came from a source in the alternative source population. Therefore, f(e u \9. M ( i) = f(e u \9 ao ). 

Note 5. It should be noted that f(e a \9 a ^) cancels from the value of evidence. This means that 
since 9 ag is known, e a is irrelevant to the resulting value of the evidence. 


Note 6. The definition of the posterior belief of (j> given X is 7r((/>|x) 


/(3#)7tQ) 

/(*) 


f f{x\(j))lT((j))d(j)' 


Note 7. The definition of posterior predictive belief of Y given X is 7r(y|a;) = / f(y\(j})ir((j)\x)d(j). 


By assuming we know (or we are certain that we know) the value 9 aQ , the denominator reduces 
to evaluating the sampling distribution of e M ; in effect the denominator does not contain any belief 
measures when the alternative source population parameters are known. We will refer to 


(3.3) 


Vss{e\9 ao ) 


^"(Cn|Cs, Tip, /) 
f(e u \9a 0 ) 


as the factored form of the specific source value of evidence when 9 a is known. 


3.3. Unknown Alternative Source Population Parameters. In this section, we propose a factor¬ 
ization of Uss that does not assume that 9 a is known, but that is still computationally tractable. 
Let e = {e s , e u , e a } represent the realization of the random element E for a specific case at hand. 
Let 9 = {@ s , 9a\. Due to the uncertainty in the parameters 9 we will need to characterize our belief 
about it using the prior. Let tt{9) = 7r(0 s )7r(0 a ) be the probability distribution used to describe our 
prior belief about 9. Note that we are choosing to restrict ourselves to priors on 9 S and 9 a that are 
independent of each other. Starting from the value of the evidence given by Equation 3.2, and using 
similar methods as the previous case, we can rewrite 14 s as follows: 
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V ss (e;9 s ,9 a ) 


ir(e\M p ) 

7r(e| M d ) 

f f(e\9,M p )n(9\M p )d9 
f f(e\9,M d )n(9\M d )d9 
f f(e u \0, M p )f(e s \8, M p )f(e a \9, M p )n(9)d9 
J f(e u \9, M d )f(e s \6, M d )f(e a \9 , M d )7r(9)d9 

I f{eu\9s)f(es\0s)f(e a \0a)Tr(6)d6 
f f(e u \d a )f(e s \e s )f(e a \da)TT(0)dd 

f f{e u \9 s )f(e s \9 s )ir(9 s )d9 s J f(e a \9 a )-K(9 a )d9 a 
f f(e s \9 s )Tr(9 s )d9 s f f (e a \9 a )f (e u \9 a )7r(9 a )d9 a 
f f(e u \9 s )f(e s \9 s )7r(9 s )d9 s J f(e a \9 a )7r(9 a )d9 a 

f f(e s \9 s )ir(9 s )d9 s * f f (e a \9 a )f(e u \9 a )Tr (9 a )d9 a 

f f(e u \9 s )f(e s \9 s )ir(9 s )d9 s / f f(e a \9 a )f(e u \9 a )n(9 a )d9 a 
J f(e s \9 s )7T(9 s )d9 s / f f(e a \9 a )ir(9 a )d9 a 

f f(e u \9 s )Ti(9 s \e s )d9 s 
f f{e u \9 a )ir{9 a \e a )d9 a 

^"(Cu|Cs) ) 

7r(e„| e a ,M d ) 


see Note 1 above 
see Note 2 above 
see Note 3 above 
see Note 8 below 
see Note 9 below 


see Note 6 above 
see Note 7 above 


Note 8. The parameters for E s and E a are fixed so they will be the same for both M p and 
Mfj. Under the prosecution model, M p , E u is characterized by 9 S since the prosecution believes 
the specific source is the origin of the trace. Therefore, f(e u \9,M p ) = f(e u \9 s ). Under the defense 
model, M d , E u is characterized by 9 a since the defense believes the trace came from a source in the 
alternative source population. Therefore, f(e u \9, M d ) = f(e u \9 a ). 


Note 9. Since tt(9) = Tr(9 s )ir(9 a ), which means that 9 S and 9 a are independent, we can factor 
the integral apart with respect to 9 into the product of the integrals for 9 S and 9 a . 


We will refer to 
(3.4) 


_ ir(e u \e Sl M p ) 
n(e u \e a ,M d ) 


as the factored form of the specific source value of evidence when 9 a is unknown. 


3.4. Results of the Factored Forms for the Value of Evidence. When the value of evidence is 
given by Equation 3.2, the computation involves evaluating the likelihood of the entire set of 
evidence E. This can be very computationally intensive, and often unfeasible. In the factored forms 
given by Equation 3.3 and Equation 3.4, the computation only involves the posterior predictive 
belief of the unknown evidence. Therefore, the factored forms create a computationally feasible 
method of evaluating the Bayes Factor for the specific source identification problem that can be 
calculated using Monte Carlo integration techniques [Kass and Raftery (1995)]. When the specific 
source value of evidence is factored into the form given by Equation 3.3 (the case when 9 ag is known 
or estimated from the data), e a is irrelevant to the value of the evidence. This considerably simplifies 
the computational complexity of the calculation of I4 S ; however, the uncertainty in the estimation 
of 9 a is not formally accounted for. This uncertainty is taken into account when the value of evidence 
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is calculated using the factorization presented in Equation 3.4 (the case when the parameters for the 
alternative source population are unknown). In this case, Monte Carlo integration techniques have 
to be used for the calculation of the denominator. This involves an additional layer of computational 
complexity; however, modern computers can cope with the required number of computations in a 
reasonable amount of time. 

4. Glass Example. In order to compare the value of the evidence obtained using both factor¬ 
izations of V ss we use a collection of samples of glass fragments studied by Aitken and Lucy (2004). 
The dataset consists of three classes of windows, with 16, 16, and 30 windows in each class. There 
are 5 glass fragments per window. Following Aitken and Lucy (2004), we consider the logarithm of 
the measurements for the ratios of elemental compositions on each glass fragment: log (Ca/K) is 
represented by the second variable (V2), log (Ca/Si) is represented by the third variable (V3), and 
log (Ca/Fe) is represented by the fourth variable (V4). 

As an illustrative example of computing the value of evidence for the specific source identification 
problem we focus only on the first class of 16 windows, and we consider two scenarios. For the first 
scenario, e u and e s will share a fixed source, with the 4 th window playing the role of the specific 
source. The first three fragments from window 4 will serve as e s and the last two fragments from 
window 4 will serve as e u . For the second scenario, e u and e s will have different sources. The first 
three fragments from window 4 will serve as e s and two fragments from the 2 nd window will serve as 
e u . In both scenarios, the remaining 70 glass fragments divided among the 14 remaining windows 
will serve as e a ■ The pairwise scatter plots of the evidence under each scenario can be seen in 
Figure 1. 

The specific source identification question for this example can stated as 
“Did the glass fragments of unknown source come from the fourth window?” 

The resulting forensic hypotheses are 

H p : The glass fragments from the unknown source came from the fourth window. 

Hg: The glass fragments from the unknown source did not come from the fourth window, but from 
some other window. 

The corresponding sampling models are 

M p : The glass fragments from the unknown source came from the fourth window. Under this 
model, E u can be characterized by the same model which characterizes E s , described in 
detail below. 

Mg: The glass fragments from the unknown source came from a randomly selected window in the 
alternative source population. Under this model, E u can be characterized by the same model 
which characterizes E a , described in detail below. 

4.1. Sampling Model for the Evidence from the Specific Source. First, we will assume that the 
measurements on the glass fragments composing e s are an independent and identically distributed, 
abbreviated i.i.d., sample from a multivariate normal with a mean vector y s and covariance T, s . Let 
y s j denote the vector of measurements on the j th fragment from the specific source evidence for 
j = 1, 2 ,m (for this example m = 3), then y s j follows a multivariate normal distribution with 
a mean vector, y s , and a covariance matrix, E s , which we will denote as y s j ~ MVN(p s ,E s ). For 
this example, the specific source population parameters are 6 S = {/r s ,S s }, so we need to specify 
priors for both y s and E s . We will use a relatively non-informative multivariate normal prior on 
y s with the zero mean vector and a diagonal covariance matrix with diagonal elements equal to 
3000. The prior for is an Inverse Wishart distribution (denoted VU _1 ) centered at a diagonal 
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Pairwise Scatter Plot of Glass Data for Scenario 1 

M2 vs V4 



V2 


Pairwise Scatter Plot of Glass Data for Scenario 2 
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Fig 1. 7n f/ie pairwise scatterplots of the evidence E, the blue asterisks represent E u , the red diamonds represent E s , 
and the gray dots represent E a . The large black dots are the mean values for each window and the gray lines show the 
deviation from that mean for each fragment from the window. 


covariance matrix, $ with diagonal elements of 0.01, 0.00005, 0.0005 and three degrees of freedom. 
These diagonal elements were chosen based on the approximate precision of the measurements for 
the evidence. The full model for E s with supporting prior beliefs is summarized below. 

Usj ~ MVN(/i s , £ s ) 
p s ~ MVN(0, 3000/) 

It should be noted that any number of reasonable priors can be chosen for gt s and £ s . The 
numerator for the values of evidence n(e u \e s , M p ) under scenario 1 ( Exp 1 ) and scenario 2 (Exp 2) 
are given in Table 1 and Table 2 below. 

4.2. Sampling Model for the Evidence in the Alternative Source Population. Next, we assume 
that the measurements on the glass fragments composing e a follow a hierarchical multivariate 
normal model with the assumption that all windows in the alternative source population have a 
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mean p a , a within-covariance matrix and a between source covariance matrix Let denote 
the vector of measurements on the j th fragment, for j = 1, 2,..., m; (for this example, m = rrii = 5 
for all i) from the i th window, for i = 1,2, ...,n (for this example, n = 14). The hierarchical 
multivariate model in this case is a simple random effects model where the between-source effects 
a* are i.i.d. multivariate normal random vectors with a mean vector of zero and a covariance 
matrix E&. The within-source effects Wij are assumed to be i.i.d. multivariate normal vectors with 
a mean vector of zero and a covariance matrix of E w . We will compare the results of both scenarios 
described above under two different conditions. First, the alternative source population parameters 
are assumed to be known. Secondly, the alternative source population parameters are assumed to 
be unknown. 

When the alternative source population parameters 6 a = {p a , E&, are assumed to be known, 
we use the estimates for the parameters as suggested in Aitken and Lucy (2004). The estimates 
d a = {p a , X&, E w } of the known parameters are summarized below. 


f^a 




ran 


i=lj=l 


m E 


m r- , 

3 = 1 


— 


n(m _ n E - yi)(ya - &) T 

' ' i= 1 j =1 


E/, = 


n — 1 


E(y* ~ Aa)(y* - Aa) 5 


2—1 



r i - i 

— 



m 


The denominator for the values of evidence f(e u \9a 0 ) under scenario 1 (Exp 1) and scenario 2 
(Exp 2) are given in Table 1 below. 


Table 1 

Alternative Source Population Parameters Known 



Exp 1 

Exp 2 

7T (e u \e 3 , M p ) 
f(eu\0a o ) 
Vss(e\0a n ) 

119740.3 

582.6974 

205.4931 

2.316277 

144.1683 

0.01606648 


When the alternative source population parameters 9 a = {p a , £&, are unknown, our prior 
for is the same that is used for £ s , and the prior for p a is the same as that used for p s . We 
use an Inverse Wishart prior for £& centered at the identity covariance matrix with three degrees 
of freedom. The full model for E a with supporting prior beliefs is summarized below. 

For z = 1,2,..., 14 and j = 1, 2,3,4, 5 : 

yij = Pa T CLi + Wij Pa ~ MVN (0,30007) 

Oi ~ MVN(0, £ b ) S 6 ~ IF _1 (I, 3) 

u>ij ~ MVN(0, S w ) S w ~ IT^, 3) 

The denominator for the values of evidence vr(e u |e s ,M p ) under scenario 1 (Exp 1) and scenario 
2 (Exp 2) are given in Table 2 below. 

The computations were performed on a 2012 MacBook Pro with an OS X 10.8.5 operating 
system, 2.3 GHz Intel Core i7 processor, and 16 GB, 1600 MHz memory using R version 3.0.2. 
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Table 2 

Alternative Source Population Parameters Unknown 



Exp 1 

Exp 2 

w(e u |e s , M p ) 
7r(e u |e a , M d ) 
V ss {e) 

119740.3 

30.17140 

3968.669 

2.316277 

209.5902 

0.01105146 


All posterior predictive distributions for the parameters are estimated using the “MCMCglmm” 
package in R [Hadfield (2010)]. Using the parameter values sampled from these posterior predictive 
distributions, we estimated the posterior predictive beliefs of e u for the values of evidence using 
a standard Monte Carlo average integration technique. The Monte Carlo estimates were based 
on a sample size of 29,000 (30,000 iterations with a burn-in of 1000). We also made use of the 
matrix form of the multivariate simple random effects models as presented by Miller (1977) (See 
the Appendix for details). 

5. Discussion. In the illustrative example described in Section 4, the behavior of V ss is con¬ 
sistent between the calculations when the alternative source population parameters are assumed 
to be known (results for this experiment can be found in Table 1) and when the alternative source 
population parameters are unknown (results for this experiment can be found in Table 2). The value 
of evidence for the first scenario suggests in both cases that the evidence is more likely to have 
been generated according to the prosecution model than by the defense model (since the values of 
evidence are significantly greater than one); the value of the evidence for the second scenario sug¬ 
gests in both cases that the evidence is more likely to have arisen under the defense model. These 
results were expected by the design of the experiment. However, by contrasting the likelihood of the 
evidence under the defense model in Table 1 and Table 2, we observe that in the first scenario the 
evidence is more likely when the alternative source population parameters are known than when 
they are not, while in the second scenario the evidence is more likely when there is uncertainty on 
the alternative source population parameters. When the number of observations in the alternative 
source population is small, there is a marked difference between these values which suggests that 
using the estimates of the parameters is not a reasonable surrogate for calculating the value of the 
evidence in the presence of uncertainty in the alternative source population parameters. We expect 
that the difference between V ss (e',8 a ) and V ss (e) will go to zero as the amount of evidence about 
the alternative source population becomes large (the rate of convergence is currently being investi¬ 
gated by the authors). In practice, the use of estimates of the known parameters for the alternative 
source population may lead to grossly over- or under-estimating the value of the evidence. The 
direction of the misleading effect will depend upon the rarity of the characteristics of e u in the 
alternative source population. This effect may ultimately mislead the criminal justice system. 

It should be noted that choosing different priors for the model parameters can results in radically 
different values of evidence. Additionally, when the alternative source population parameters are 
unknown there is less freedom in choosing the priors than when the parameters are known. In order 
for the factorization of the value of evidence to hold when there is uncertainty in the alternative 
source population parameters, the prior for the specific source parameters, 7 t( 9 s ), must be chosen 
to be independent from the prior for alternative source parameters, n(0 a ). This precludes the use 
of the popular “random man” prior for the specific source parameters in which the specific source 
is believed to be typical of the alternative source population. 

6. Conclusion. Computing the value of evidence when it is given by its raw form (Equa¬ 
tion 3.2) requires evaluating the likelihood of the entire set of evidence. In most situations, this 
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evaluation is computationally unfeasible. However, when the value of evidence is given in the fac¬ 
tored forms (Equation 3.3 and Equation 3.4) it can be approximated using Monte Carlo integration 
techniques [Kass and Raftery (1995)]. In the rare setting when there is no uncertainty about the 
alternative source population parameters (Equation 3.3), it is not surprising that including e a in 
the calculation of the value of evidence has no impact. However, in traditional forensic settings, 
there is rarely sufficient information about the alternative source population parameters to assume 
that it is possible to estimate them accurately; henceforth, such situations require a novel approach. 

In this paper, we develop a logical and coherent method which formally incorporates the uncer¬ 
tainty on the alternative source population parameters into the calculation of the Bayes Factor. 
This is a major departure from the ad-hoc approaches to include uncertainty about the background 
population in the value of the evidence that are currently available in the forensic statistics liter¬ 
ature. These methods typically entail the construction of a confidence or credible interval for the 
likelihood ratio represented by V^ s (e; 9 ao ). By formally incorporating uncertainty about e a into the 
value of evidence we can guarantee that the resulting value is statistically rigorous and that the 
decisions based on it will be admissible in a statistical decision theoretic sense. To avoid potentially 
misleading decisions in the court system, the authors suggest replacing the use of ad-hoc methods 
with the statistically rigorous methods presented here when there is uncertainty in the alternative 
source population parameters. 


APPENDIX A 

We will summarize a well-known result from Miller concerning the matrix form of the multi¬ 
variate simple random effects model [Miller (1977)]. Let yjj denote the A;-dimensional vector of 
measurements on the j th component from the i th source for j = 1,2, ••• , rrii and i = 1,2, ...,n. 
Then the simple random effects model is given by 


Vij — ft a T CLi + Wij 

where Nk{ 0, X a ) and Wij Np( 0, E m ) are independent from each other. It is often useful 

to think about the random effects model as a combined multivariate normal random ^vector. If 

= (yn Ui2 ••• Vim^j has the 


yij follows a simple random effects model, then the vector Yi 
following distribution: 


Yi ~ N k (n c , E c ) 



^Yi a + Y w E a • • • X a ^ 


(70 


XL XL + XL, ' • • : 



where X c = 

*—‘CL A —‘CL 1 

^ Rq • • • Ra Ra + Rui ) 

and r c = 

w 
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